58 research outputs found
Efficiency versus Convergence of Boolean Kernels for On-Line Learning Algorithms
The paper studies machine learning problems where each example is described
using a set of Boolean features and where hypotheses are represented by linear
threshold elements. One method of increasing the expressiveness of learned
hypotheses in this context is to expand the feature set to include conjunctions
of basic features. This can be done explicitly or where possible by using a
kernel function. Focusing on the well known Perceptron and Winnow algorithms,
the paper demonstrates a tradeoff between the computational efficiency with
which the algorithm can be run over the expanded feature space and the
generalization ability of the corresponding learning algorithm. We first
describe several kernel functions which capture either limited forms of
conjunctions or all conjunctions. We show that these kernels can be used to
efficiently run the Perceptron algorithm over a feature space of exponentially
many conjunctions; however we also show that using such kernels, the Perceptron
algorithm can provably make an exponential number of mistakes even when
learning simple functions. We then consider the question of whether kernel
functions can analogously be used to run the multiplicative-update Winnow
algorithm over an expanded feature space of exponentially many conjunctions.
Known upper bounds imply that the Winnow algorithm can learn Disjunctive Normal
Form (DNF) formulae with a polynomial mistake bound in this setting. However,
we prove that it is computationally hard to simulate Winnows behavior for
learning DNF over such a feature set. This implies that the kernel functions
which correspond to running Winnow for this problem are not efficiently
computable, and that there is no general construction that can run Winnow with
kernels
Translating between Horn Representations and their Characteristic Models
Characteristic models are an alternative, model based, representation for
Horn expressions. It has been shown that these two representations are
incomparable and each has its advantages over the other. It is therefore
natural to ask what is the cost of translating, back and forth, between these
representations. Interestingly, the same translation questions arise in
database theory, where it has applications to the design of relational
databases. This paper studies the computational complexity of these problems.
Our main result is that the two translation problems are equivalent under
polynomial reductions, and that they are equivalent to the corresponding
decision problem. Namely, translating is equivalent to deciding whether a given
set of models is the set of characteristic models for a given Horn expression.
We also relate these problems to the hypergraph transversal problem, a well
known problem which is related to other applications in AI and for which no
polynomial time algorithm is known. It is shown that in general our translation
problems are at least as hard as the hypergraph transversal problem, and in a
special case they are equivalent to it.Comment: See http://www.jair.org/ for any accompanying file
Redundancy, Deduction Schemes, and Minimum-Size Bases for Association Rules
Association rules are among the most widely employed data analysis methods in
the field of Data Mining. An association rule is a form of partial implication
between two sets of binary variables. In the most common approach, association
rules are parameterized by a lower bound on their confidence, which is the
empirical conditional probability of their consequent given the antecedent,
and/or by some other parameter bounds such as "support" or deviation from
independence. We study here notions of redundancy among association rules from
a fundamental perspective. We see each transaction in a dataset as an
interpretation (or model) in the propositional logic sense, and consider
existing notions of redundancy, that is, of logical entailment, among
association rules, of the form "any dataset in which this first rule holds must
obey also that second rule, therefore the second is redundant". We discuss
several existing alternative definitions of redundancy between association
rules and provide new characterizations and relationships among them. We show
that the main alternatives we discuss correspond actually to just two variants,
which differ in the treatment of full-confidence implications. For each of
these two notions of redundancy, we provide a sound and complete deduction
calculus, and we show how to construct complete bases (that is,
axiomatizations) of absolutely minimum size in terms of the number of rules. We
explore finally an approach to redundancy with respect to several association
rules, and fully characterize its simplest case of two partial premises.Comment: LMCS accepted pape
Quantum machine learning: a classical perspective
Recently, increased computational power and data availability, as well as
algorithmic advances, have led machine learning techniques to impressive
results in regression, classification, data-generation and reinforcement
learning tasks. Despite these successes, the proximity to the physical limits
of chip fabrication alongside the increasing size of datasets are motivating a
growing number of researchers to explore the possibility of harnessing the
power of quantum computation to speed-up classical machine learning algorithms.
Here we review the literature in quantum machine learning and discuss
perspectives for a mixed readership of classical machine learning and quantum
computation experts. Particular emphasis will be placed on clarifying the
limitations of quantum algorithms, how they compare with their best classical
counterparts and why quantum resources are expected to provide advantages for
learning problems. Learning in the presence of noise and certain
computationally hard problems in machine learning are identified as promising
directions for the field. Practical questions, like how to upload classical
data into quantum form, will also be addressed.Comment: v3 33 pages; typos corrected and references adde
A New Algorithm for Learning Range Restricted Horn Expressions
A learning algorithm for the class of range restricted Horn expressions is presented and proved correct. The algorithm works within the framework of learning from entailment, where the goal is to exactly identify some pre-fixed and unknown expression by making questions to membership and equivalence oracles. This class has been shown to be learnable in previous work. The main contribution of this paper is in presenting a more direct algorithm for the problem which yields an improvement in terms of the number of queries made to the oracles. The algorithm is also adapted to the class of Horn expressions with inequalities on all syntactically distinct terms where further improvement in the number of queries is obtained
- âŠ